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INTRODUCTION: 


Spinal  cord  injury  (SCI)  often  leads  to  both  the  loss  of  ability  to  move  ones  limbs  and  the  loss  of 
sensation  from  the  limbs.  One  key  component  of  this  lost  sensation  is  proprioception,  the  feeling  of 
where  the  body  is  in  space.  The  importance  of  proprioception  is  often  not  appreciated;  without  it,  we  are 
unable  to  move  normally.  Even  if  there  were  therapies  that  could  restore  movement  to  spinal  cord 
injured  patients,  without  proprioception  those  movements  will  be  slow,  clumsy  and  uncoordinated.  The 
goal  of  this  work  is  to  restore  proprioception  for  these  individuals.  In  particular,  we  are  focusing  on 
restoring  proprioception  in  the  context  of  brain  machine  interfaces  (BMIs),  in  which  neural  activity  from 
the  brain’s  motor  centers  is  monitored  and  used  to  guide  control  of  an  assistive  device  such  as  an 
orthotic  limb.  We  are  working  to  develop  and  test  a  “bi-directional”  BMI,  which  both  monitors  neural 
activity  in  motor  areas  of  the  cerebral  cortex  and  delivers  artificial  proprioceptive  feedback  via 
intracortical  microstimulation  (ICMS)  to  a  somatosensory  cortical  area.  Since  these  precise  neural 
patterns  needed  to  evoke  the  correct  proprioceptive  sense  is  not  known  in  advance,  we  are  focusing  on 
the  brain’s  ability  to  learn  to  interpret  new  signals.  In  previous  work,  we  showed  that  the  brain  can  learn 
to  interpret  arbitrary  patterns  of  ICMS  activation  and  can  use  those  patterns  to  guide  movement 
(Dadarlat,  O’Doherty,  and  Sabes,  Nature  Neuroscience,  2015).  We  are  working  to  extend  this  approach 
to  a  bi-directional  BMI. 


2.  KEYWORDS: 

Spinal  cord  injury;  brain-machine  interfaces;  artificial  feedback;  proprioception;  somatosensation; 
microstimulation;  movement  control 

3.  ACCOMPLISHMENTS: 

What  were  the  major  goals  of  the  project? 

Specific  Aim  1 :  Determine  whether  animals  can  learn  to  use  artificial  proprioception 

Artificial  proprioception  is  delivered  The  ICMS  feedback  signal  will  not  at  first  be  meaningful  to  the 
animals.  However,  that  signal  will  correlate  on  a  millisecond  timescale  with  visual  feedback  of  the  virtual 
limb.  Based  on  the  previous  work  (Dadarlat  et  al.,  2015),  we  expect  these  correlations  to  drive 
naturalistic  integration  of  ICMS.  After  learning,  we  will  use  behavioral  measures  to  determine  how  well 
the  animal  can  interpret  the  ICMS  signal,  alone  and  combination  with  visual  feedback.  We  will  determine 
whether  ICMS  and  vision  are  integrated  in  a  minimum-variance  manner,  as  expected  for  “natural” 
sensory  signals. 


•  Major  T ask  1 . 1 :  Train  animals  in  basic  procedures  (Months  1  -6) 

o  This  task  is  complete. 

•  Major  Task  1.2:  Experiment  1  -  data  collection  (Months  1-13) 

o  Monkey  1 :  In  our  first  experiment  with  Monkey  1 ,  we  found  that  the  animal’s  performance 
with  artificial  proprioception  (delivered  via  ICMS)  and  vision  was  improving,  compared  to 
that  achieved  with  vision  alone.  The  pace  of  learning  was  comparable  to  our  earlier 
published  study.  However,  as  reported  in  the  quarterly  report,  we  encountered  a  technical 
setback:  the  electrode  array  in  Ml  of  Monkey  1  failed,  no  longer  providing  robust  enough 
single  unit  recordings  to  provide  high-quality  BMI  control.  We  have  therefore  explanted  the 
array  in  this  animal.  We  will  implant  new  arrays  in  the  next  quarter. 

o  Monkey  2:  From  our  experience  with  Monkey  1 ,  we  are  confident  that  Monkey  2  can  be 
transitioned  to  brain  control  quickly  after  the  implantation  of  electrode  arrays.  As 
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previously  reported,  we  decided  to  delay  implantation  of  Monkey  2  in  order  to  take  into 
account  the  results  of  the  main  experiment  with  Monkey  1 .  Because  the  electrode  arrays 
have  finite  longevity  (see  above),  this  will  maximize  the  use  of  that  time  for  Monkey  2.  We 
plan  to  implant  Monkey  2  in  the  next  quarter  as  well. 

•  Major  Task  1 .3:  Experiment  1  -  data  analysis  (Months  4-22) 

o  We  have  made  considerable  progress  on  data  analysis,  and  expect  to  be  nearly  on  target 
with  this  task,  despite  the  delays  in  Major  Task  1 .2. 

o  We  have  made  use  of  the  time  afforded  by  the  experimental  delays  to  address  a  major 
challenge  raised  by  the  use  of  electrical  stimulation.  Stimulation  creates  a  large  electrical 
artifact  that  interferes  with  simultaneous  recordings  from  motor  cortex.  We  originally  dealt 
with  this  issue  by  temporally  separating  the  stimulation  and  recording  intervals.  We  have 
now  developed  powerful  new  tools  for  removing  the  artifact  from  the  recording  signals  (see 
below). 


Specific  Aim  2:  Determine  whether  artificial  proprioception  improves  BMI  learning  rate  and  asymptotic 
performance 

Whether  or  not  naturalistic  integration  is  achieved  in  Aim  1 ,  we  expect  that  the  addition  of  feedback 
signals  directly  to  SI  will  improve  BMI  performance.  We  will  measure  the  learning  rate  for  a  new  BMI 
controller,  the  asymptotic  performance  of  that  controller,  and  the  long-term  stability  of  control,  and 
compare  these  measures  for  cases  with  and  without  artificial  proprioceptive  feedback. 


•  Major  Task  2.1 :  Experiment  2  -  data  collection  (Months  10-30) 

o  The  start  of  these  experiments  has  been  delayed  due  to  delays  in  Major  Task  1 .2.  We  still 
expect  to  be  able  to  accomplish  the  key  elements  of  this  experiment  by  the  end  of  the 
funding  period. 

•  Major  Task  2.2:  Experiment  2  -  data  analysis  (Months  1 2-36) 

o  This  task  will  begin  shortly  after  the  start  of  Experiment  2. 


What  was  accomplished  under  these  goals? 

Major  Tasks  1.2-1 .3:  Experiment  1 

Monkey  1  performed  approximately  400  “training”  trials  a  session  (i.e.,  per  day).  These  training  trials 
consisted  of  BMI  reaches  with  combined  visual  feedback  and  artificial  proprioceptive  feedback.  The 
artificial  feedback  conveys  the  two-dimensional  location  of  the  BMI-controlled  cursor  via  intracortical 
microstimulation  stimulation  (ICMS).  The  ICMS  signal  consists  of  multiple  bi-polar  pairs  of  electrodes 
(one  acting  as  cathode,  one  as  anode)  delivering  sequences  of  charge-balanced,  biphasic  current 
pulses.  The  frequency  of  the  stimulation  pulses  is  varied  as  a  function  of  cursor  position  in  the  2-D 
space,  with  5  independent  “channels”  of  directional  information  (i.e.,  using  a  redundant  five  basis  vectors 
to  encode  two-dimensional  space).  The  goal  of  these  training  trials  was  to  allow  the  animal  to  learn  to 
interpret  the  ICMS  signal  and  integrate  it  into  the  relevant  sensorimotor  circuits. 

We  monitored  progress  toward  that  goal  with  a  smaller  number  of  “testing”  trials,  with  a  variety  of 
feedback  conditions.  The  most  important  comparison  during  this  early  stage  of  learning  is  between 
testing  with  visual  feedback  alone  and  testing  with  both  visual  feedback  and  artificial  feedback  (“STIM”). 
Results  for  the  first  experiment  are  shown  Figure  1 ,  for  two  performance  metrics:  average  absolute 
Cartesian  endpoint  error  and  average  absolute  angular  error  in  initial  movement  angle  (with  respect  to  a 
straight  line). 
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Figure  1.  Differences  between  bimodal  (VIS+STIM)  and  vision-only  (VIS)  trials,  as  a  function  of 
testing  session,  for  two  performance  metrics.  Smaller  values  equal  better  performance,  so  differences 
less  than  zero  translate  to  better  performance  with  the  stimulation  feedback  on.  p-values  are  for 
regression  slopes. 

Early  in  training,  performance  on  both  metrics  was  impaired  by  stimulation  (p<0.001 , 2-way  ANOVA  on 
session  and  feedback  condition),  but  relative  performance  on  the  VS  condition  improved  overtime.  By 
the  last  10  sessions,  VIS+STIM  trials  had  significantly  lower  endpoint  error  (p<0.03;  not  corrected  for 
multiple  comparisons).  It  is  notable  that  it  took  about  8,000  training  trials  (20  sessions)  until  performance 
improved  with  stimulation.  However  this  results  is  on  par  with  our  earlier  published  study:  improvement 
with  feedback  was  not  observed  until  about  13,000-15,000  trials,  depending  on  the  error  metric  (see 
Dadarlat,  O’Doherty  and  Sabes,  Nature  Neuroscience,  2015).  From  that  study,  we  expect  that  with  more 
training  (-30-40  additional  sessions)  the  animal  would  have  been  able  to  make  BMI  reaches  without 
vision,  using  only  stimulation  feedback. 

This  experiment  was  aborted  due  to  electrode  failure,  and  we  recently  explanted  the  original  arrays. 
During  the  explantation  surgery,  we  noted  that  that  the  SI  array  was  only  superficially  in  the  brain, 
having  been  pushed  out  by  granulation  tissue  (as  is  commonly  seen:  Barrese,  et  al. ,  J  Neural  Eng. 

2013,  10(6):066014),  which  explains  why  we  needed  to  stimulate  multiple  electrodes  with  high  currents 
(100  uA)  to  elicit  performance  above  chance  at  our  preliminary  detection  task.  We  will  restart  training  in 
both  animals  once  new  arrays  are  implanted  in  the  coming  quarter,  and  we  expect  that  better  array 
placement  will  result  in  better  overall  performance  with  the  artificial  ICMS  feedback.  Furthermore,  we  will 
take  advantage  of  the  restart  to  improve  the  experimental  approach,  in  particular  with  the  use  of  ICMS- 
artifact  rejection,  allowing  for  continuous  control  and  feedback  (instead  of  the  interleaved  control  and 
feedback  used  in  the  first  experiment). 

Major  Tasks  1 .2-1 .3:  ICMS  Artifact  Rejection 

A  focus  of  our  analysis  has  been  on  techniques  to  reject  stimulation  artifact,  so  that  we  can  perform 
continuous  stimulation  and  recording  for  BMI  control.  We  have  developed  two  approaches  to  solve 
this  problem.  First,  as  shown  in  Figure  2,  we  now  can  perform  online  artifact  template  subtraction 
from  multiple  channels. 

In  a  second  approach,  we  use  an  adaptive  filter  that  takes  advantage  of  our  multielectrode 
recordings.  In  particular,  for  each  electrode,  we  predict  the  stimulation  artifact  on  that  electrode  by 
adaptively  filtering  the  signals  recorded  on  all  of  the  remaining  electrodes.  This  provides  a 
remarkable  robust  artifact  rejection  scheme  that  generalizes  across  many  different  stimulation  sites 
(Figure  3A)  and  performs  even  with  highly  non-stationary  artifacts  (Figure  3B).  Furthermore,  only 
about  10-20  recording  sites  are  needed  to  achieve  nearly  asymptotic  performance  (Figure  3C). 
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Figure  2.  Stimulation  artifact  removed  from  multiple  independent  and  simultaneous  current  sources, 
even  when  pulses  completely  overlap.  (A)  Pulse  trains  (top)  were  delivered  to  four  electrodes  in  primary 
somatosensory  cortex  of  a  rhesus  macaque,  each  with  its  own  independent  pulse  repetition  rate.  The 
stimulation  artifact  contaminated  concurrent  recordings  (middle)  made  in  primary  motor  cortex.  The 
artifact  was  modeled  as  linear  and  additive  and  was  successfully  removed  (bottom)  allowing  action 
potentials  to  be  detected.  (B)  The  efficacy  of  the  additive  model  is  demonstrated  by  generating  pulse 
trains  (top)  on  multiple  channels  of  stimulation  at  different  temporal  offsets  with  respect  to  each  other. 
The  resulting  artifacts  (middle)  can  be  completely  eliminated  in  many  cases  or  greatly  reduced  in 
amplitude  (bottom;  note  scale).  Shaded  region  indicates  standard  deviation.  (C)  The  residual  signal 
after  artifact  removal  is  at  the  noise  floor  (dashed  line)  for  many  pulse  offsets.  For  completely 
overlapping  pulses  the  residual  signal,  while  larger  than  the  noise  floor,  is  smaller  than  the  amplitude  of 
a  typical  extracellular  action  potential. 
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Figure  3.  Stimulation  artifact  using  adaptive  multi-channel  filtering.  (A)  Performance  of  the  adaptive 
filter  to  remove  a  sequence  of  pulse  trains  delivered  on  different  channels  (1-8),  showing  that  the  same 
filter  parameters  work  no  matter  which  channels  of  stimulation  are  used.  (B)  The  example  of  stimulation 
channel  5,  at  an  enlarged  scale,  shows  good  artifact  removal  despite  highly  non-stationary  artifact 
(presumably  from  a  faulty  stimulator  channel).  (C)  Performance  of  the  adaptive  filtering  method  as  a 
function  of  the  number  of  electrodes  used,  showing  that  performance  nears  asymptotic  levels  with  only 
a  small  number  of  recording  electrodes  (-10-20). 


What  opportunities  for  training  and  professional  development  has  the  project  provided? 

Nothing  to  Report. 


How  were  the  results  disseminated  to  communities  of  interest? 

Nothing  to  Report. 


What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 

Our  effort  in  the  next  year  will  be  focused  on: 

•  Major  Task  1 .2  and  1 .3 

o  Implanting  Monkeys  1  &  2 

o  Completing  Experiment  1  with  both  monkeys,  using  the  improvements  described  above 
o  Publishing  our  artifact  rejection  scheme 

•  Major  Task  2.1  and  2.2 

o  Starting  Experiment  2,  after  completion  of  data  collection  for  Experiment  1 


4.  IMPACT: 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

•  We  have  developed  powerful  new  tools  for  removing  artifacts  from  electrical  recordings  from  the 
brain  due  to  simultaneous  brain  microstimulation.  We  anticipate  that  these  tools  will  be  widely 
used  in  the  field  and  will  have  a  substantial  impact  on  the  improvement  of  bi-directional  BMIs,  i.e. 
devices  that  combine  neural  stimulation  and  recording. 
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•  Our  preliminary  results  suggest  that  the  use  of  artificial  sensory  feedback  delivered  via  brain 
microstimulation  will  improve  performance  of  a  BMI,  compared  to  performance  with  visual 
feedback  alone.  If  these  results  are  replicated  and  expanded  in  upcoming  experiments,  we 
believe  the  impact  for  the  BMI  community  will  be  great.  In  particular,  this  work  would  show  that  it 
is  possible  to  obtain  performance  benefits  even  when  it  is  not  possible  to  be  able  to  replicate  the 
patterns  of  activity  that  would  have  occurred  before  spinal  cord  injury. 

What  was  the  impact  on  other  disciplines? 

•  The  artifact  removal  scheme  will  have  impact  beyond  BMI  applications.  For  example,  the  scheme 
will  be  useful  for  “causal”  neuroscience  experiments,  in  which  stimulation  is  used  to  study  the 
dynamics  of  brain  circuits. 

What  was  the  impact  on  technology  transfer? 

Nothing  to  report 

What  was  the  impact  on  society  beyond  science  and  technology? 

Nothing  to  report 


5.  CHANGES/PROBLEMS: 

Changes  in  approach  and  reasons  for  change 

Nothing  to  report 

Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

As  reported  in  previous  quarterly  reports,  we  have  experienced  delays  due  the  failure  of  the  electrode 
implants  in  Monkey  1 .  We  have  already  explanted  those  arrays  and  plan  to  implant  new  arrays  in  the 
coming  quarter.  While  this  has  delayed  progress  on  Specific  Aim  1 ,  it  has  also  given  us  time  to  improve 
the  experimental  approach,  as  described  in  detail  above.  We  anticipate  that  we  will  finish  Aim  1  in  the 
coming  year. 

Changes  that  had  a  significant  impact  on  expenditures 

Our  spend  rate  was  higher  than  originally  planned  in  Year  1 .  The  difference  was  due  to  the  extra  initial 
effort  contributed  by  Drs.  Sabes  and  Hanson  in  order  to  get  the  project  started.  We  do  not  expect  that 
this  accelerated  burn  rate  will  persist  into  second  year. 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards,  and/or 
select  agents 

Significant  changes  in  use  or  care  of  human  subjects 

Nothing  to  report  (not  applicable) 

Significant  changes  in  use  or  care  of  vertebrate  animals. 

Nothing  to  report 
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Significant  changes  in  use  of  biohazards  and/or  select  agents 
Nothing  to  report 
6.  PRODUCTS: 

Publications,  conference  papers,  and  presentations 

Nothing  to  report 

Website(s)  or  other  Internet  site(s) 

Nothing  to  report 
Technologies  or  techniques 

As  described  above,  we  have  developed  powerful  new  tools  for  removing  artifacts  from  electrical 
recordings  from  the  brain  due  to  simultaneous  brain  microstimulation.  We  expect  that  these  tools  will 
find  wide  application.  We  are  currently  preparing  a  manuscript  describing  these  tools,  which  will  be 
submitted  soon.  The  manuscript  will  provide  sufficient  information  for  other  groups  to  readily  employ 
these  techniques. 

Inventions,  patent  applications,  and/or  licenses 

Nothing  to  report 

Other  Products 

Nothing  to  report 


7.  PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 
What  individuals  have  worked  on  the  project? 

Provide  the  following  information  for:  (1)  PDs/PIs;  and  (2)  each  person  who  has  worked  at  least  one 
person  month  per  year  on  the  project  during  the  reporting  period,  regardless  of  the  source  of 
compensation  (a  person  month  equals  approximately  160  hours  of  effort).  If  information  is  unchanged 
from  a  previous  submission,  provide  the  name  only  and  indicate  "no  change." 


Name: 

Philip  Sabes,  PhD 

Project  Role: 

Principal  Investigator 

Researcher  Identifier  (e.g. 
ORCID  ID): 

orcid.org/0000-0001  -8397-6225 

Nearest  person  month 
worked: 

3 

Contribution  to  Project: 

Dr.  Sabes  is  the  PI.  He  has  provided  supervision  and 
leadership  for  all  aspects  of  the  project 

Funding  Support: 

1.  DARPA/Case  Western,  iSens:  Implanted  somatosenso 
electrical  neurostimulation 
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2.  DARPA,  Unlearning  neural  systems  dysfunction  in 
neuropsychiatric  disorders 

3.  DARPA,  A  new,  scalable  approach  to  high-bandwidth, 
minimally  invasive  neural  recording  and  stimulation 

Name: 

Joseph  O’Doherty 

Project  Role: 

Postdoc 

Researcher  Identifier  (e.g. 
ORCID  ID): 

orcid.org/0000-000 1  -8 1 75-5699 

Nearest  person  month 
worked: 

1  (Dr.  O'Doherty  is  also  supported  by  a  PVA  fellowship  wi 
largely  overlapping  aims.  Therefore,  Dr.  O'Doherty  is 
collectively  contributing  100%  effort  to  the  Aims  of  this 
project.) 

Contribution  to  Project: 

Dr.  O’Doherty  has  been  principally  responsible  for 
performing  the  experiments  and  analyses  in  this  project. 

Funding  Support: 

1 .  Postdoctoral  fellowship,  Paralyzed  Veterans  of 

America 

Name: 

Timothy  Hanson 

Project  Role: 

Development  Engineer 

Researcher  Identifier  (e.g. 
ORCID  ID): 

Nearest  person  month 
worked: 

4 

Contribution  to  Project: 

Mr.  Hanson  has  provided  technical  support  for  all  aspects 
of  this  project 

Funding  Support: 

Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key  personnel  since 
the  last  reporting  period? 

Sabes  Active: 

W9 1 1 N  F- 1 5-2-0054  5/1 5/20 1 5-6/ 14/2017 

DARPA 

A  new,  scalable  approach  to  high-bandwidth,  minimally  invasive  neural  recording  and  stimulation 

N6600 1  - 1 5-C-40 14  1/1 6/20 1 5-4/28/20 1 6 

Case  Western  Reserve  University  (DARPA) 

iSens:  Implanted  somatosensory  electrical  neurostimulation  and  sensing 
The  latter  of  these  is  new  in  the  last  year. 

What  other  organizations  were  involved  as  partners? 

Nothing  to  report 


8.  SPECIAL  REPORTING  REQUIREMENTS 


See  attached  Quad  Chart. 

9.  APPENDICES: 

Nothing  to  report 
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Restoring  Proprioception  via  a  Cortical  Prosthesis:  A  Novel  Learning-Based  Approach 
SCI  30074 
W81XWH-1 4-1 -0510 

PI:  Philip  N.  Sabes  Org:  UCSF  Award  Amount:  $710,683 


Study/Product  Aim(s) 

•  Determine  whether  animals  can  learn  to  use  artificial 
proprioception  delivered  via  intracortical  microstimulation  (ICMS) 
to  primary  somatosensory  cortex 

•  Determine  whether  artificial  proprioception  improves  brain- 
machine  interface  (BMI)  learning  rate  and  asymptotic  performance 

Approach 

A  key  factor  that  has  limited  performance  of  neuroprostheses  is  the 
lack  of  natural  proprioceptive  feedback.  Our  objective  is  to 
develop  a  learning-based  approach  for  providing  artificial 
proprioception,  taking  advantage  of  the  brain’s  capacity  for 
plastic  reorganization. 


Timeline  and  Cost 


Activities  CY 

14 

15 

16 

17 

1.1:  Train  animals  in  basic 
procedures 

1 

1 .2:  Expt.  1  -  artificial  feedback  with 
learned  BMI  control:  data  collection 

M 

1 .3:  Expt.  1  -data  analysis 

mm 

1 .2:  Expt.  2  -  artificial  feedback 
while  learning  BMI  control:  data 
collection 

— 

1 .3:  Expt.  2  -data  analysis 

Estimated  Budget  ($K) 

$55 

$189 

$257 

$211 

Updated:  29  October  2015 


<u 


E 


Our  refined  our  artifact  rejection  scheme  removes  stimulation-induced  recording 
artifacts  due  to  multi-electrode  stimulation.  This  will  permit  a  much  more  efficient 
closed-loop  control  scheme  with  continuous  and  simultaneous  feedback  and  control. 

Goals/Milestones 

CY14  Goal  -  Behavioral  Training 

1ZI  Preliminary,  basic  behavioral  training 

CY15  Goals  -  Demonstrate  artificial  feedback  with  learned  BMI 

□  Perform  Experiment  1  with  Monkey  1 

□  Perform  Experiment  1  with  Monkey  2 

CY16  Goal  -  Begin  simultaneous  learning  of  artificial  feedback  and  BMI 

□  Complete  Experiment  1  and  prepare  manuscript 

□  Perform  Experiment  2  with  Monkey  1 

□  Perform  Experiment  2  with  Monkey  1 

CY17  Goal  -  Obtain  improved  learning  and  performance  with  feedback 

□  Complete  Experiment  1  and  prepare  manuscript 

Comments/Challenges/lssues/Concerns 

•  Need  to  re-implant  Monkey  1 ,  but  artifact  rejection  will  make  for  a 
more  efficient  closed-loop  control  scheme 
Budget  Expenditure  to  Date 
Projected  Expenditure:  $193,230 

Actual  Expenditure:  $193,230 


